Image processing in current digital camera systems is done either by using a hardware (HW) implementation, a software (SW) implementation or their combination thereof. The HW implementation can be integrated to a camera head, a system IC (integrated circuit) or be a separate component. The HW implementation is typically fast and can achieve high frame rates for real time operations like viewfinder and video. The downside of the HW implementation is the added HW cost and inflexibility in regard to image quality tuning needs.
On the other hand the SW implementation of the image processing pipeline may be cost-effective and flexible, and may be tuned easily. However, a typical constraint in the SW based implementation is available processing power or MIPS (million instructions per second, a processor performance measure). Real time operations like video recording and displaying the image on a viewfinder in the SW based image processing pipeline may be especially problematic. This is because there is a fixed time budget and all operations including data transfer from camera (e.g., from an image sensor) to a display have to be done in given amount of time. For example, in order to achieve a frame rate of 15 frames per second (fps), only 66.7 milliseconds (ms) is available to perform all operations, and to achieve 30 fps only 33 ms is available, etc. To achieve such a fast processing may be problematic in low cost processor systems (especially for low-light video applications), e.g., with desired higher picture/screen resolutions.
This problem with SW implementation may be partially reduced by optimizing the image processing SW, i.e., by dropping certain image processing operations, e.g., like lens shading correction. Optimizations may be possible to a certain extent but there is a limit to what degree the processing time can be reduced because dropping image processing operations will degrade the image quality.
Another approach may be using downscaling/upscaling for all frames in the image. In this method a lot of image information may be lost and the image quality may be noticeably degraded.